I like the principle of indifference. However, I think infinities are unfalsifiable in principle. In this case, does it make sense for me to attribute probabilities to them? If I did, they would be just metaphysical priors that can never be updated by any evidence.

St Petersburg-like lotteries, defined in terms of your Bayesian credences, don't require assigning positive probability to any possible infinities out there in the world.

I'll leave the rest in a footnote, because it's not that relevant to the point I've been making so don't plan to go further with it, but I already wrote it and it may be of interest to you.^[1]

1. ^{^}
   1. I think falsifiability for a Bayesian could just mean we can imagine evidence that would warrant assigning very very low credences to the hypothesis. Actual infinities are often falsifiable in principle in this way, in specific cases. I think the probability you should assign to you being infinite in spatial size is extremely small. Similarly for the Earth being infinite in size.
   2. In the case of the spatial extent of the universe being infinite, it seems hard to falsify now only because all the evidence we've gathered so far is in fact consistent with it (or no less consistent than with a bounded/finite universe), and it remains one of the standard models used by experts. My own view is that it’s more likely than not infinite in spatial extent, because it’s the simplest model consistent with the evidence.
   3. I think our credences that the universe is infinite in spatial extent should increase with our credences that the universe is globally flat (0 global curvature), which has been measurable. If we had good evidence of nonzero global curvature, that would be decent (but not overwhelming) evidence that the universe is finite/bounded, because it would rule out the most plausible infinite models of the universe.
   4. I agree that we can sometimes compare actions with infinite expected cost-effectiveness. I'm most partial to expansionist views of some kind, as the most complete (given precise probabilities).

Thanks for doing the work to make a specific ALLFED cost effectiveness estimate! I think the AIs made a number of good points. However, I was saying the CEARCH result is ~$170 per life because GiveWell uses $5000/life and CEARCH was saying 30x as cost effective as GiveWell. I think GiveWell uses averting a child death means saving ~37 DALYs, and an adult death ~30 DALYs. I don't think the AI's assumption of 80 QALYs per life saved is realistic (unless you are expecting radical life extension). The AI starts with CEARCH and then adjusts cost per life saved upward. I think there are good reasons why CEARCH is an overestimate of cost per life saved. For one, it finds nuclear risk to be significantly smaller than volcanic risk. Most analysts in this space think that the nuclear risk is significantly larger than the volcanic risk. Furthermore, the AI assumes that ALLFED's work outside of policy for abrupt sunlight reduction scenario (ASRS) is less cost-effective than the ASRS policy work. However, I think the pandemic work is likely to be even more cost-effective, especially from the long term perspective, because pandemics are generally regarded as a greater existential risk.

The AI did seem to agree with the argument that ALLFED should have a long-term impact, it just didn't think that the AI x-risk estimate should be used. That's fine - I didn't think you would want to do a bespoke model for ALLFED, but now that you have done it for the near term, I do think it is important to do it for the long term. The AI points out that the marginal cost effectiveness calculations of the longterm impact in the journal articles are out of date because we have now spent more money. Of course that's true, but that's why we also calculated the cost effectiveness of spending hundreds of millions of dollars to see if the whole effort was justified. And indeed that still came out as more cost effective than AI safety. Now of course other things have changed since ~2021. AI timelines have gotten much shorter, but we were assuming that only $3 billion would be spent on AI safety, and I think it's pretty clear that a lot more than that will be spent now (especially if you count the total compensation including stock options of AI safety workers in the labs (even with your weighting of 0.3 for lab work), but that might be a topic for another post). AI 2040 hopes that trillions of dollars will be spent on AIS. Also since then, nuclear risk has gotten larger per year with the Ukraine war and potential acceleration and destabilization due to AI. Also, engineered pandemic risk per year has gone up with AI capabilities. However, this does mean a shorter number of years in expectation that the nuclear and pandemic risk might be relevant if you think the nuclear and pandemic risk will go away after AGI/ASI. For comparison, your cost per microprobability of reduction in x-risk of AI safety is $1.2 million. The median in the papers for the 3 billionth dollar on AIS was $2.5 million, with the mean being lower, so pretty good agreement with your value. So overall, since 2021, the relative marginal cost effectiveness of spending hundreds of millions of dollars on GCR resilience vs what we think will be spent on AIS I don't think has changed too much. 

The AI missed other outside evaluations of ALLFED's longterm impact:

Speedrun: Demonstrate the ability to rapidly scale food production in the case of nuclear winter by Marie Buhl from Rethink Priorities: "my (extremely rough) estimate that this project reduces x-risk with a cost-effectiveness of ~$260 million per 0.01% absolute reduction^[1] (~70% confidence interval: 2.2 million to 2.7 billion). If this estimate were accurate, then this project would clear our median roughly estimated cost-effectiveness bar of $500M per basis-point of x-risk averted".

Shallow evaluations of longtermist organizations by Nuño Sempere: "I disagree strongly with ALLFED's estimates (probability of cost overruns, impact of ALLFED's work if deployed, etc.), however, I feel that the case for an organization working in this area is relatively solid." (Note that this is a 5 year old analysis, but he recently said he respects ALLFED more now).

I'm curious if you have thoughts on how it'd be if you submitted feedback like you've done here into some sort of form, the LLMs went back and forth processing it like that, updated the site, and published a transcript like the above. I think if I make it fully automated right now it'd be fairly exploitable due to LLM sycophancy, unless I tried pretty hard to mitigate that.

Interesting idea! I guess it would be less exploitable than direct edits like Wikipedia.

-

As for putting 'unknown' EAs/rationalists on the list, I see the drawback of putting a lot of them on if you are targeting a general audience. But I do think it is compelling to show that even without a long-term perspective, donating to existential risk reduction can allow everyday people to beat billionaires in terms of lives saved.

Your prior that the problem is not anything other than low pay should be incredibly strong. If, as is prognosticated, a huge sum of money flows into AIS, then Google senior staff engineers and VPs will be in play to lead research projects, and I strongly suspect the prerequisite that the candidate "must be AIS-pilled" will quietly resolve itself. Pricey management consultants at McKinsey and BCG will be retained to speed all this up.

My contrarian stance: veganism is not EA. The impact of veganism on your time, money, health, budget, enjoyment of veganism is substantial. You're likely doing useful, impactful things with your time and energy when you have them, and veganism is a drain on your time and energy.

The only thing that tips the scale toward veganism in my judgment (I do not presently practice veganism), is the practice just interests you. Despite the above detriments, most people can go vegan if they want. If you love it, feel empathetic to animals, feel it refreshes your values, sure, do it.

But it's not the sort of practice I would make central to EA. It should be as taboo as playing guitar, basketball, or backpacking, which is to say not taboo, but not automatically on topic.

Reddit chatter is like:

• hate datacenters, would support just about any anti-dc regulation
• if you believe anything about the HF incident or any AI capabilities, you are a shill falling for AI marketing

I think both views are propagandized. My mental model is, propaganda hardens and narrows political beliefs, and I think on both points, the social media consensus has gone beyond what widespread sincere belief looks like.

The distilled factual version is really not too hard for ordinary people to understand. "The models are very powerful and they can't control them." If people are just never really getting to that, it's not complexity, it's some other psychological or social force. (I concede the big picture conclusion which is like "oh and it can attempt take over," sounds way more absurd to an AIS-novice than the constituent parts.) Any propaganda researchers reading? I'd like to know WTF is going on with point #2 above.

Recently, I thought you could take something like anti-dc sentiment and attach wonkish AIS riders. I think I'm right that negative AI valence is very important, but I'm more concerned propaganda can interfere with "valence => desired action."

On the other hand -- a  number of elites seem to have noticed what happened, e.g. D Ted Lieu and R Nathaniel Moran with AI Kill Switch Act.